refactor: align multimodal example port allocation with vLLM components #4163

nnshah1 · 2025-11-06T18:55:18Z

Summary

Replace dynamic port allocation with environment variable-based configuration to match components/src/dynamo/vllm/args.py approach
Remove runtime.allocate_port_block() in favor of DYN_VLLM_KV_EVENT_PORT environment variable
Simplify port management to be consistent across the codebase

Changes Made

Port Allocation Strategy

Removed dynamic port allocation via runtime.allocate_port_block()
Added environment variable-based configuration using DYN_VLLM_KV_EVENT_PORT
Added port validation with registered port range (1024-49151)

Configuration Functions

Updated configure_ports() to be synchronous and use environment variables
Added create_kv_events_config() function to match VLLM components pattern
Updated overwrite_args() to follow the same structure as VLLM components

Host Management

Added ensure_side_channel_host() and get_host_ip() functions
Removed the set_side_channel_host_and_port() function
Updated to only set host (not port) for NIXL side channel

Code Structure

Removed all port allocation metadata tracking
Simplified Config class by removing side_channel_port field
Updated worker.py to call non-async configure_ports(config)

Test plan

Verify multimodal examples still work with environment variable port configuration
Test that DYN_VLLM_KV_EVENT_PORT environment variable is properly respected
Confirm NIXL side channel host resolution works correctly
Validate port range validation (1024-49151) functions as expected

🤖 Generated with Claude Code

Summary by CodeRabbit

Chores
- Updated port configuration mechanism to use environment-driven allocation instead of runtime-based allocation, simplifying port management for multimodal examples.
- Refined side-channel host management with improved default handling and fallback logic.
- Streamlined configuration by removing unnecessary side-channel port-related settings and adding conditional port requirements based on feature enablement.

…in multimodal example Align multimodal example port configuration with components/src/dynamo/vllm/args.py approach. Changes: - Replace allocate_and_reserve_port() with get_kv_port() using DYN_VLLM_KV_EVENT_PORT env var - Update configure_ports() to be synchronous and environment-based - Replace set_side_channel_host_and_port() with ensure_side_channel_host() (host only) - Remove side_channel_port from Config class - Update worker.py to use new configure_ports(config) signature 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>

coderabbitai · 2025-11-06T19:20:16Z

Walkthrough

Port configuration logic is refactored across two files. The async port allocation mechanism is replaced with synchronous environment-driven retrieval. Function signatures are updated, and the Config class no longer holds side_channel_port. Several port-allocation functions are removed and replaced with new configuration helpers.

Changes

Cohort / File(s)	Change Summary
Port Configuration API Refactoring `examples/multimodal/utils/args.py`	Replaced async port allocation with synchronous `get_kv_port()` function using environment variables. Removed `side_channel_port` field from `Config` class and functions `allocate_and_reserve_port()` and `set_side_channel_host_and_port()`. Added `ensure_side_channel_host()` helper, `configure_ports(config)` function, and `DEFAULT_ENDPOINT` constant. Updated `overwrite_args` to conditionally require `kv_port` only when `enable_prefix_caching` is enabled.
Port Configuration Call Site `examples/multimodal/components/worker.py`	Updated `configure_ports()` invocation from `await configure_ports(runtime, config)` to `configure_ports(config)`, removing the `runtime` parameter and eliminating the `await` keyword.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Verify async-to-sync conversion is correct and does not introduce race conditions
Validate environment variable fallback logic (particularly in get_kv_port() and ensure_side_channel_host())
Confirm all call sites of removed functions (allocate_and_reserve_port, set_side_channel_host_and_port) are updated or no longer needed
Check that conditional kv_port requirement logic in overwrite_args correctly gates on enable_prefix_caching
Ensure DEFAULT_ENDPOINT usage is consistent throughout the codebase

Poem

🐰 Ports once danced in async dreams so grand,
Now environment speaks, a simpler plan!
No more runtime queues, just variables bright,
Configuration flows—synchronous and light! ✨

Pre-merge checks

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main refactoring change: aligning multimodal example port allocation with vLLM components, which is the core objective of the PR.
Description check	✅ Passed	The PR description comprehensively covers all template sections including overview, detailed changes, and related context, though it doesn't reference a specific GitHub issue.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

examples/multimodal/utils/args.py (1)
139-170: Fix missing kv_port initialization when prefix caching default is None

AsyncEngineArgs.enable_prefix_caching comes out of CLI parsing as None until vLLM applies its defaults.(docs.vllm.ai) With the new synchronous flow, that falsy value skips the configure_ports branch and leaves config.kv_port unset. As soon as overwrite_args formats the ZMQ endpoint, config.kv_port - dp_rank raises a TypeError, so the worker never starts.

Set config.kv_port before we build the defaults by treating None as “we will enable prefix caching” (which matches the V1 default) and assigning the port eagerly.
 def configure_ports(config: Config):
     """Configure port settings from dedicated environment overrides."""
 
-    if config.engine_args.enable_prefix_caching:
-        config.kv_port = get_kv_port()
+    enable_prefix = config.engine_args.enable_prefix_caching
+    if enable_prefix is None:
+        enable_prefix = True
+        config.engine_args.enable_prefix_caching = True
+
+    if enable_prefix:
+        config.kv_port = get_kv_port()
 
     ensure_side_channel_host()

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between e0fb52d and 0c3cae8.

📒 Files selected for processing (2)

examples/multimodal/components/worker.py (1 hunks)
examples/multimodal/utils/args.py (1 hunks)

🧰 Additional context used

🧬 Code graph analysis (2)

examples/multimodal/components/worker.py (1)

examples/multimodal/utils/args.py (1)

configure_ports (139-145)

examples/multimodal/utils/args.py (4)

lib/bindings/python/examples/hello_world/server_sglang_static.py (1)

Config (31-37)

lib/bindings/python/examples/hello_world/server_sglang.py (1)

Config (40-46)

lib/bindings/python/examples/hello_world/server_sglang_tok.py (1)

Config (43-49)

lib/bindings/python/examples/hello_world/server_vllm.py (1)

Config (52-58)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)

GitHub Check: trtllm (amd64)
GitHub Check: trtllm (arm64)
GitHub Check: vllm (arm64)
GitHub Check: operator (arm64)
GitHub Check: sglang (amd64)
GitHub Check: sglang (arm64)
GitHub Check: vllm (amd64)
GitHub Check: Build and Test - dynamo

examples/multimodal/utils/args.py

Update launch scripts to set unique VLLM_NIXL_SIDE_CHANNEL_PORT values for multi-worker deployments to avoid port conflicts. Uses ports 20097, 20098, 20099 to match the pattern from components folder examples. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>

The overwrite_args function always uses config.kv_port regardless of prefix caching settings, so configure_ports must always set this value to prevent "NoneType and int" TypeError. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <[email protected]>

pull-request-size bot added the size/L label Nov 6, 2025

github-actions bot added the refactor label Nov 6, 2025

nnshah1 force-pushed the neelays/multi-modal-fixes branch from 3992615 to 6aabff9 Compare November 6, 2025 19:04

copy-pr-bot bot temporarily deployed to GITLAB November 6, 2025 19:04 Inactive

copy-pr-bot bot temporarily deployed to GITLAB November 6, 2025 19:09 Inactive

nnshah1 force-pushed the neelays/multi-modal-fixes branch from 6aabff9 to 0c3cae8 Compare November 6, 2025 19:12

copy-pr-bot bot temporarily deployed to GITLAB November 6, 2025 19:13 Inactive

nnshah1 marked this pull request as ready for review November 6, 2025 19:13

nnshah1 requested review from a team as code owners November 6, 2025 19:13

copy-pr-bot bot temporarily deployed to GITLAB November 6, 2025 19:14 Inactive

coderabbitai bot reviewed Nov 6, 2025

View reviewed changes

examples/multimodal/utils/args.py Show resolved Hide resolved

nnshah1 and others added 2 commits November 6, 2025 11:27

copy-pr-bot bot temporarily deployed to GITLAB November 6, 2025 22:07 Inactive

copy-pr-bot bot temporarily deployed to GITLAB November 6, 2025 22:08 Inactive

nnshah1 force-pushed the neelays/multi-modal-fixes branch from 316141e to c0da5bd Compare November 7, 2025 00:13

nnshah1 requested review from alec-flowers, grahamking and kthui November 7, 2025 00:24

Merge branch 'main' into neelays/multi-modal-fixes

aa07b36

copy-pr-bot bot temporarily deployed to GITLAB November 7, 2025 01:20 Inactive

copy-pr-bot bot temporarily deployed to GITLAB November 7, 2025 01:21 Inactive

grahamking approved these changes Nov 7, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

refactor: align multimodal example port allocation with vLLM components #4163

refactor: align multimodal example port allocation with vLLM components #4163

nnshah1 commented Nov 6, 2025 •

edited by coderabbitai bot

Loading

Uh oh!

coderabbitai bot commented Nov 6, 2025

Uh oh!

coderabbitai bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

refactor: align multimodal example port allocation with vLLM components #4163

Are you sure you want to change the base?

refactor: align multimodal example port allocation with vLLM components #4163

Conversation

nnshah1 commented Nov 6, 2025 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes Made

Port Allocation Strategy

Configuration Functions

Host Management

Code Structure

Test plan

Summary by CodeRabbit

Uh oh!

coderabbitai bot commented Nov 6, 2025

Walkthrough

Changes

Estimated code review effort

Poem

Pre-merge checks

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

nnshah1 commented Nov 6, 2025 •

edited by coderabbitai bot

Loading